FusionPillars: A 3D Object Detection Network with Cross-Fusion and Self-Fusion

In the field of unmanned systems, cameras and LiDAR are important sensors that provide complementary information. However, question how to effectively fuse data from two different modalities has always been a great challenge. this paper, inspired by idea deep fusion, we propose one-stage end-to-end network named FusionPillars multisensor (namely point cloud camera images). It includes three branches: point-based branch, voxel-based an image-based branch. We design modules enhance voxel-wise features in pseudo-image: Set Abstraction Self (SAS) fusion module Pseudo View Cross (PVC) module. For single sensor, considering relationship between point-wise features, SAS self-fuses branch spatial information pseudo-image. sensors, through transformation images’ view, PVC introduces RGB as auxiliary cross-fuses pseudo-image image scales supplement color Experimental results revealed that, compared existing current networks, yield superior performance, with considerable improvement detection precision for small objects.